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I.  INTRODUCTION 


Suppose  d  is  a  class  of  information  sources.  For  a  given  distortion 
measure,  consider  the  function 

sup  R  (D);  D  ^  0  (1) 

a€tf  a 

where  R  (D)  is  the  rate-distortion  function  (corresponding  to  the  given 
distortion  measure)  for  the  individual  source  a.  The  quantity  in  (1)  is 
of  interest  in  the  study  of  coding  schemes  for  classes  of  sources.  For 
example,  under  compactness  conditions  on  the  class  d  the  quantity  in  (1) 
represents  a  rate -distortion  function  ford  (as  defined  by  Sakrison  [1]),  and, 
even  for  noncompact  classes,  the  quantity  of  (1)  is  a  lower  bound  on  this 
rate-distortion  function.  Further,  if  there  is  a  universal  code  for  d, 
then  the  quantity  in  (1)  describes  the  worst-case  rate  (versus  distortion) 
required  to  transmit,  via  the  universal  code,  an  arbitrarily  chosen  member 
of  d. 

In  this  paper  we  consider  the  function  in  (1)  for  classes  of  homogeneous 
discrete-parameter  sources  that  are  specified  only  in  terms  of  the  spectral 
properties  of  their  elements.  We  consider  the  particular  version  of  (1) 
corresponding  to  the  single -letter  mean-square-error  (MSE)  distortion 
measure.  Note  that  for  the  one -parameter  case,  which  corresponds  to  d 
being  a  class  of  covariance-stationary  discrete-time  sources  specified  only 
in  spectral  terms,  the  MSE  version  of  the  function  in  (1)  is  unchanged  if 
d  is  replaced  by  its  subset  consisting  only  of  Gaussian  sources  (Berger 
[2,  p.  154]).  Thus,  in  this  context,  it  is  reasonable  to  restrict  attention 
to  the  consideration  of  classes  of  Gaussian  sources.  Such  a  class  d  can 
be  specified  completely  by  defining  a  class  17{  of  spectral  measures,  and  the 
determination  of  the  quantity  of  (1)  for  two  specific  source  classes  of 
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this  type  has  been  considered  previously  by  Sakrison  [3].  Here,  for  a 
general  type  of  such  classes,  we  demonstrate  the  existence  of  a  member 
spectrum  whose  rate-distortion  function  achieves  (1)  for  each  value  of 
D  >  0.  In  particular,  we  consider  classes  77\  whose  upper  measures  are 
Choquet  alternating  capacities  of  order  2  (Choquet  [4]).  Such  classes 
occupy  a  central  position  in  generalizations  of  the  theories  of  hypothesis 
testing  (Huber  and  Strassen  [5])  and  .of  stationary  linear  smoothing  (Poor 
[6]),  and  include  many  important  classes  such  as  contaminated  mixtures, 
variation  neighborhoods,  and  Prohorov  neighborhoods.  We  show  here  that 
the  spectrum  achieving  the  supremum  in  (1)  for  such  a  class  is  given  by 
the  derivative  (in  the  sense  of  Huber  and  Strassen  [5])  with  respect  to 
Lebesgue  measure  of  the  upper  measure  of  771  and  corresponds  to  the  element 
of  771  that  is  closest  to  Lebesgue  measure  in  a  sense  defined  by  directed 
divergence.  Since  Lebesgue  measure  represents  a  white  spectrum  (corres¬ 
ponding  to  a  memory less  source),  the  maximizing  spectrum  is  thus  the  member 
of  77\  that  is  "most  memoryless",  a  result  that  certainly  agrees  with  the 
intuitive  meaning  of  the  MSE  rate-distortion  function. 

Section  II  contains  a  more  complete  specification  of  the  spectral 

classes  to  be  considered  and  gives  properties  of  these  classes  that  are 

relevant  to  the  determination  of  the  quantity  sup  R  (D) .  A  number  of 

a€  Cl  a 

examples  are  also  presented  that  demonstrate  the  generality  of  this  type 
of  class.  Section  III  contains  the  main  analytical  results  concerning  the 
quantity  in  (1)  for  such  classes.  In  particular  the  existence  of  a  maxi¬ 
mizing  spectrum  is  demonstrated,  and  the  characterization  of  this  spectrum 
as  the  element  of  %  closest  to  Lebesgue  measure  is  established.  Section 
IV  considers  in  detail  the  specific  case  in  which  the  class  771  consists  of 
all  spectra  that  are  a  convex  mixture  of  a  discrete-time  wide-sense  Markov 
spectrum  and  an  unknown  "contaminating"  spectrum,  and  the  results  of 
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Section  III  are  illustrated  directly  for  this  case.  The  extension  of  the 
results  to  continuous-parameter  cases  is  discussed  in  Section  V. 


II.  CHOQUET  CAPACITIES,  HUBER-STRASSEN  DERIVATIVES,  AND  SPECTRAL  UNCERTAINTY 

Throughout  this  paper  n  is  a  fixed  positive  integer  and  X  *  {Xt;t€Zn} 
is  ann-parameter  homogeneous  Gaussian  source  (Z  denotes  the  set  of  all 
integers).  By  way  of  Bochner's  theorem  (Wong  [7,  p.  245])  a  class  of 
sources  of  this  type  can  be  specified  by  defining  a  class  of  spectral 
measures  on  (Q,£)  where  fi  denotes  the  n-dimensional  rectangle  [-TT,rr]n  and 
3  denotes  the  Borel  tj-algebra  on  Q.  In  this  paper  we  will  restrict  our 
attention  primarily  to  classes  %  satisfying  the  power  constraint  m(0)  *  (2TT)"nP 
for  all  m  €771  where  P  is  a  fixed  positive  number.  It  is  straightforward  to 
relax  this  constraint;  this  restriction,  however,  allows  us  to  consider  the 
effects  of  spectral  shape  on  the  rate-distortion  function  in  more  detail. 

The  upper  measure  v  of  a  class  IK  with  power  constraint  is  the  set 
function  on  B  defined  by 

v(B)  ■  sup  m(B);  B  6  3.  (2) 

m€  7K 

Note  that  v  has  the  following  properties:  (i)  v(0)  =»  0  and  v(fi)  *  (2rr)  nP, 

(ii)  A  c  B  implies  v(A)  S  v(B),  and  (iii)  B  t  B  implies  v(B  )  t  v(B),  where 

n  n 

all  sets  are  assumed  to  be  in  3  and  0  denotes  the  null  set.  If  IK  is  weakly 
compact,  then  v  has  the  additional  property  (Huber  and  Strassen  [5,  p.  252]): 
(iv)  Fn  1  F  with  Fr  closed  for  all  n  implies  v(Fn)  1  v(F).  A  set  function 
having  properties  (i)  through  (iv)  is  a  capacity  on  3  in  the  sense  of 
Choquet  [4].  In  this  paper  we  consider  classes  whose  upper  measures  satisfy 
(i)-(iv)  and  the  additional  property:  (v)  v(A  U  B)  +  v(A  fl  B)  £  v(A)  +  v(B) 
for  all  A,  B  €  8.  A  capacity  satisfying  this  latter  property  is  said  to  be 
alternating  of  order  2  and  is  termed  a  2-altemating  capacity.  Note  that  a 
finite  measure  is  an  example  of  a  2-altemating  capacity. 
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If  the  upper  measure  v  of  a  class  17[  is  a  2 -alternating  capacity  then 
%.  coincides  with  the  class  defined  by 

*  [m  €  m|  m(B)  £  v(B);  B  €  8,  and  m(0)  *  v(p)}  (3) 

where  M  denotes  the  class  of  all  finite  measures  on  (fl,8 )  (Huber  and 

Strassen  [5,  Lemma  2.3]).  Classes  of  the  form  of  (3)  include  many  of  the 

traditional  models  for  spectral  uncertainty,  and  a  number  of  useful  examples 

are  discussed  below.  It  is  interesting  to  note  that  all  such  classes  are 

weakly  compact  [3,  Lemma  2.2]  and  that,  if  v  is  a  measure,  then#?  ■  N). 

Thus  any  results  obtained  for  ^  also  apply  to  a  single  measure;  in  fact 

the  classes  ^  have  been  useful  In  generalizing  hypothesis  testing  and 

Wiener  filtering  results  which  hold  for  single  measures  (such  as  the 

Neyman-Pearson  Lemma)  to  classes  of  measures  [3,6]. 

The  properties  of  classes  of  the  form  of  (3)  have  been  studied  by 

Huber  and  Strassen  [5].  The  properties  of  such  classes  relevant  to  the 

present  problem  can  be  summarized  in  the  following  two  lemmas  (here,  and 

elsewhere  in  this  paper,  X  denotes  Lebesgue  measure  on  (P^)): 

Lemaa  1:  Suppose  v  is  a  2-alternating  capacity  on  (p,/3 ).  There  exists  a 

Lebesgue-measurable  function  tt  :fl  “*  [0,«]  such  that^w,,  ([tt  >9})*  inf  w0  (B) 

V  9  v  B€£ 

for  each  3  2  0  where  Wg  is  the  set  function  on  8  defined  by 

wq(B)  -  v(B°)  +  0X(B),  B €8.  (4) 

Furthermore,  rrv  is  unique  a.e.  [X]. 

*For  compactness  of  notation  we  will  write  {f  >  9}  to  denote  (<a€  Q  [  f  (cu)  >8  }  • 
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Proof :  Since  X  is  a  finite  measure  and  hence  a  2-alternating  capacity  on 
( V.J3 ),  the  existence  of  ttv  follows  from  Lemmas  3.1  and  3.2  of  [5].  The 
uniqueness  of  tt^  follows  from  Theorem  5.1  of  [5]. 

Lemma  2:  Suppose  v  and  rr  are  as  in  Lemma  1  and  *  S"tt  >0],  Y  9  i  0. 

V  u  V 

c  c 

Then  there  exists  a  measure  q  9  ^  such  that  ttv  *  dq/dX  and  q(^)  *  v(Bq) 
for  all  9^0.  Here  dq/dX  denotes  the  generalized  Radon-Nikodym  derivative 
of  q  with  respect  to  X;  that  is,  dq/dX  may  be  infinite  on  a  set  of  X 
measure  zero. 

Proof :  The  existence  of  such  a  q  follows  from  Theorem  4.1  of  [5]  and  from 
the  construction  in  the  proof  of  this  theorem  (see  also  Huber  and  Strassen 
[8]). 

Note  that,  if  v  is  a  finite  measure,  then  the  function  rr^  is  the 
(generalized)  Radon-Nikodym  derivative  of  v  with  respect  to  X.  Partly  for 
this  reason  Huber  and  Strassen  termed  the  function  rrv  the  Radon-Nikodym 
derivative  of  v  with  respect  to  X.  However,  to  distinguish  tt^  from  an 
ordinary  Radon-Nikodym  derivative,  we  will  term  ttv  the  Huber-Strassen 
derivative  of  v  with  respect  to  X.  Letnnas  1  and  2  are  particular  cases  of 
more  general  results  for  differentiating  one  capacity  with  respect  to 
another  [5],  In  this  more  general  context,  the  Huber-Strassen  derivative 
is  the  basis  for  minimax  hypothesis  testing  between  two  classes  of 
probability  measures  of  the  form  7 7^  [5]  and  for  minimax  linear  smoothing 
of  a  signal  with  spectral  measure  in  a  class  of  the  form  77^  observed  in 
additive  noise  with  spectral  measure  in  a  class  of  the  form  77^  [6]. 

Lemmas  1  and  2  give  the  basic  properties  needed  to  consider  the  rate- 
distortion  function  over  classes  of  Gaussian  sources  determined  by  spectral 
classes  of  the  form  of  (3).  We  conclude  this  section  by  giving  several 

useful  examples  of  such  classes  that  illustrate  the  generality  of  this 
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model.  In  Examples  1  and  2,  is  a  fixed  finite  spectral  measure  on  (pyC?) 
and  e  is  a  fixed  number  in  [0,1].  In  all  cases  v^(0)  is  defined  to  be 
zero. 


Example  1  (e-mixtures):  The  set  function  v^(B)  *  (l-e)mg(B)  +  em^Q), 

2 

b€/9,  B  f4  0  is  a  2-alternating  capacity  and  77?  is  given  by 

1 

77|v  =  {m€  m|  m*  (1  -  e)mg  + eh  for  some  h€  M  with  h(p)  *m0(Q)}.  (5) 

Thus  m^  can  be  thought  of  as  a  nominal  spectral  model  and  e  as  a  degree 
of  uncertainty  placed  on  the  model.  The  model  of  (5)  is  one  of  the  earliest 
models  used  for  uncertainty  in  robust  hypothesis  testing  and  signal- 
detection  studies  (Huber  [9],  Martin  and  Schwartz  [10])  and  was  noted  in 
[5]  to  be  of  the  form  of  (3).  The  Huber-Strassen  derivative  of  v^  with 
respect  to  Lebesgue  measure  on  Q  is  found  by  noting  that  the  set  function 
w.  of  (4)  is  given  for  this  case  by 


VB) 


(1  -  e)m0(BC) +emQp) +eX(B);  B  +  Q 
1 0X  <p  )  *  9(2TT)n  ;  B  =  n 


(6) 


which  is  minimized  over  0  by  the  set 


2 

Note  that  this  set  function  (and  the  one  defined  in  Example  2)  is  dis¬ 
continuous  fromabove  at  the  null  set  0.  However,  for  fi  -  [ -tt , tt ] n ,  this 
discontinuity  does  not  violate  the  property  that  v  must  be  continuous 
from  above  on  closed  sets  (i.e..  Property  (iv))  since,  in  a  compact 
separable  metric  space,  there  is  no  sequence  of  closed  sets  converging 
down  to  the  null  set  (Dunford  and  Schwartz  [11,  pp.  30-31]). 
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Cttq  >  8/  (1-e )}  ;  if  wQ  (£ttq  >  0/(l-e)})  <  0  ,)n 

0  ;  otherwise 


(7) 


where  TrQ  =«  dmQ/dX.  Noting  that  we  can  define  =  inf{8  ^  0|cjd?  Bg], 

we  have  that 


"v  (®)  =  max{ c 1 ,  (1  -  e)n0(«))}  ,  u)€n. 


where  c'  =  sup(0  s  o|  Wg  ({tt^  >  0/ (1  -  e)}  )  <  0  (2rr)n}  .  a  specific  example 
of  this  class  will  be  considered  in  Section  IV  below. 

Example  2  (variation  neighborhoods):  The  set  function  v^ (B)  * 
min{m0(B)  +  €mQ(P) ,  mQ(P)},  B  6  B,  B  5 4  0,  is  a  2-altemating  capacity  with 
given  by 


*  (m€  m|  P  (m,m0)  <  ,  and  m(Q)  ■  mQ(n)} 


where  p  is  the  variational  distance  (or  Kolmogorov  metric)  defined  on  M 

by  p(|i,v)  »  sup|p,(B)  -  v  (B)  |  .  Classes  of  the  form  of  (9)  have  been 
B 6/9 

considered  previously  in  the  contexts  of  robust  hypothesis  testing  (Huber 
[9])  and  robust  Wiener  filtering  (Poor  [12]).  Note 
that  this  class  is  also  a  model  for  a  degree  e  of  uncertainty  in  a 
nominal  spectral  model  m^,  although  the  deviation  allowed  in  (9)  is  somewhat 
different  from  that  allowed  in  (5).  Note  further  that  (9)  with  degree  of 
uncertainty  e/2  contains  (5)  with  degree  of  uncertainty  e.  The  de¬ 
rivative  of  v^  with  respect  to  Lebesgue  measure  on  Q  can  be  found  from 
general  results  in  [13]  and  [14]  and  is  given  by 
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tt  (us)  »  max{c'  >min{c,,,TT0(u))}}  ;  to  €  n  ,  (10) 

2 

where  rt^  =  dn^/dX,  c"  =  inf{9  s  0j tnQ ({tTq  >  9])  *  em^ (G)  <8X  ({tTq  >  8])},  and 
c'  »  sup{0  £  0|8X({tt0  S  8})  is  h1q({tt0  £  8  } )  +  em0(fi)}. 

Example  3  (band  models):  Suppose  P  >  0  is  the  power  in  X  and  that  m^ 

and  my  are  two  finite  spectral  measures  on  Q  ,8)  with  m^)  <  (2Tr)aP<  m^  (ft) . 

Define  two  2-alternating  capacities  v  and  v  by  v  (B)  =  m  (B)  + 

L  U  aj 

((2TT)nP -mL(n)),  B€5,  B^0,  and  v  (B)  =min{mlJ(B),  (2n)np}  ,  B€£,  Bj*0.  Then 
the  set  function  v_(B)  =  min£v  (B),  v  (B)3  is  a  2-alternating  capacity 

■j  J_»  u 

on  <p  ,8)  and  27L  is  given  by 
3 

*  Cm  6  M|m  <  m  and  m{l)  *  (2fr)np}  .  (11) 

L  U 

Thus ,  7f[v  for  this  case  is  the  band  of  spectral  measures  lying  between  the  spectral 
3 

measures  and  aiy.  This  class  is  known  as  the  band  model  and  has  been  utilized 
previously  in  problems  of  robust  signal  detection  (Kuznetsov  [15],  Kassam  [16])  and 
robust  Wiener  filtering  (Kassam  and  Lim  [17]),  This  class  was  shown 
to  be  of  the  type  (3)  by  Vastola  and  Poor  [18],  Note  that,  with  mL  = 

(1  -  s)m^  and  with  niy  -*  ®,  the  class  of  (11)  gives  the  e-mixture  class  of 
(5).  The  derivative  of  v^  with  respect  to  Lebesgue  measure  onH  can  be  deter¬ 
mined  from  a  result  in  [18  ]  and_ involves  both  derivatives  dm^/dX  and  dm^/dX. 

Other  examples  of  classes  of  the  form  of  (3)  include  Prohorov 
neighborhoods  of  a  nominal  spectral  measure  (see  Strassen  [19,  p«  438]) 
and  generalizations  of  the  mixture  model  of  Example  1  generated  by  mixed 
capacities  of  the  form  v(B)  »  Jv^(B)M-  (dg),  where  u  is  a  measure  on 
(see  Strassen  [19,  Theorem  4]).  A  class  of  spectral  measures  that  does 
not  fit  the  model  of  (3)  is  Sakrison's  Model  (b)  (see  [3,  p.  11])  which 


essentially  consists  of  those  measures  placing  exact  amounts  of  spectral 
measure  on  members  of  a  set  of  intervals  covering  the  interval  [  — tt , tt ] . 

This  type  of  model  is  not  of  the  type  in  (3)  because  such  classes  of  measures 
are  not  weakly  closed  (Vastola  and  Poor  [18]). 
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III.  THE  RATE-DISTORTION  FUNCTION  ON  SPECTRAL  CLASSES  GENERATED  BY 
CAPACITIES 

The  MSE  rate-distortion  function  for  a  homogeneous  Gaussian  source 

X  with  spectral  measure  m  depends  only  on  the  part  of  m  that  is  absolutely 

continuous  with  respect  to  Lebesgue  measure.  This  function,  denoted  by 

R  (D),  is  given  by  the  parametric  relationship 
m 

Rm(DQ(m))  =  (4tt) "n  f  raax{0,  Log (TTm (cju ) /0 ) } X (dou)  (12) 

ft 

and 


D0  (m) 


T  min{9,  rr  (uj)]X(du)) 

d  m 


where  tt^  =  dm/dX  is  the  generalized  Radon-Nikodym  derivative  of  m  with 
respect  to  X,and  0  is  a  parameter  ranging  over  the  interval  (O.A^]  with 
^  ■  ess  sup  tt^uo).  Note  that  0  S  4^  <  «,  since  Cl  is  compact.  The  expression 
of  (12)  and  (13)  is  given  by  Berger  [2,  Theorem  4.5.3]  for  the  case  n  *  1 
and  follows  from  Hayes,  Habibi,  and  Wintz  [20]  for  n  S£  2. 


It  follows  from  (12)  and  (13)  that  the  support  of  R  (D)  is  the 

m 

interval  (0,  D  )  and  that  R  (D  )  ■  0,  where  D  *  (2tt)  n  f  tt  dX.  With 
m  mm  m  km 

respect  to  this  latter  quantity  we  have  the  following  result: 


-  .  » 

!-;> 

jjV  «i 


>•;'  >-, 


b 

t? 
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Proof:  Suppose  m  6  57?  .  For  each  9^0,  define  secs  B*  *  frr  >9}  and 

V  W  V 

^  ■  [tt  >  9}.  By  Lemma  1  we  have 


m(^)  +9\(Ag)  -  inf  [o(BC)  +  0X(B)]  *  m(B^)  +  9X  (Bg  ) 
B  €  *5 

s  v(B^)  +  9X(B&) 


(14) 


for  each  9^0,  where  Che  second  inequaliCy  follows  from  Che  f act  chat 

m  £  v.  Lemma  2  states  that  there  is  a  measure  q  such  that  n  is  a  version 

n  v 

c  c 

of  dq/dX  and  v(Bq)  *  q(B^).  Thus,  (14)  implies 

lev**!.  V**9fc<v-*<vi 
H  h 

for  each  9  <  ®,  which  in  turn  implies 


P  tt  dX  £  T  tt  dX  +  f  tt  dX  +  f  n  dX  (15) 

J  ,c  m  J_c  v  v  J.  m  v/ 

^  ®9  ^ 

for  each  9  <  ®,  Taking  the  limit  as  9  -*  ®  in  (15)  and  defining  B  *  fl  B* 

®  9  >0  ^ 

and  A  ■  n  Aj ,  we  have 
*  9  >0  d 


* 


TTydX  + 


f 

J 


n  dX 
v 


r 

o 

4, 


ndl 

tu 


(16) 


Since  A  and  B  both  have  X  measure  zero,  (16)  is  equivalent  to  ‘  tt  dX  < 

w  00  VL  TO 

f  rr  dX  which  was  to  be  shown. 

a 

We  see  from  Leona  3  that  no  m€#?v  has  more  nonsingular  power  than 

the  meoiber  q  singled  out  by  Lensna  2.  What  we  will  show  in  the  following 

paragraphs  is  that  R  (D)  achieves  sup  R  (D),  and  hence  that  the  spectral 

q  rriE97?v  ° 

density  defined  by  nv  also  achieves  this  supremum.  To  do  this,  we  first 


13 


need  the  following  result  which  seems  intuitively  obvious  from  the 
derivation  of  (12)  and  (13)  and  from  analogous  results  for  memoryless 
sources  (see  Berger  [2,  Theorem  4.2.1  and  pp.  110-111])  but  for  which 
we  could  find  no  previously  published  proof. 

Lemma  4:  Suppose  m  is  a  spectral  measure  on  ,B )  with  MSE  rate-distortion 

function  R  (D).  Then  R  (D)  has  a  right-hand  derivative  R'(D)  for  all 
mm  m 

D€  (0,D  )  where  D  *  (2rr)  n  f  tt  dX,  and  this  derivative  is  given  by 
m  m  n  m 

R^(D)  »  -2  **8^  where  8^  is  determined  uniquely  by  the  equation 


D 


(2TT)"n 


f  min{8 
U 


,rr  (u))]X  (duu). 
m 


(17) 


Proof:  The  existence  of  the  right-hand  derivative  of  R  (D)  follows  from 

m 

the  fact  that  Rm(D)  is  convex  on  (0,Dm)  (see  Berger  [2,  p.  270]  and 

Royden  [21,  p.  109]).  The  relationship  DQ  (m)  ■  (2rr)  n[m(Crtm<9])  + 

9X ({tt^  >  ©3 ) ]  defines  a  continuous,  strictly  increasing  mapping  from 

(0,  4  )  onto  (0,  D  ).  This  follows  by  noting  that,  for  0  <  8  <  9'  <  A  , 
mm  m 

we  have 


(9'  -  9)X  (£tt  >9})  *  <B  '  -9)X((n  >  9})  -  f  (9  '  -it  )dX  - 
in  m  v  in 

•  (2tT)n[I^i(m)  -  Dg  (m)]  »  (9  '  -8  )X<£rr  >8  '  ])  +  j'(Tim-9)dX 

{e<TT  £0'} 

2  <9  '  -9)X(Ctto  >  9  ’})  >  0,  m  (18) 

where  the  final  inequality  follows  from  the  fact  that  9 '  <  L  ■ 

in 

ess  sup  tt  (uj).  Thus,  a  unique  9  is  determined  by  (17)  for  each  d€  (0,D  ), 
oj6D  m  D  m 

and  the  right-hand  derivative  of  R  (D)  is  given  by  the  expression 


R'(D)  -  lim  [(R  (Da (m))  -  R  (D))/ (D. (m)  -  D)]. 
m  n  i  a  mo  m  o 


(19) 
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Equation  (12)  implies  that  the  numerator  in  (19)  can  be  written  as 


R(D, (m))  -  R  (D) 
mo  m 


(4TT)"n[log(9  /9)X({rr  >  9})  +  J*  log (9d/tt  )dX] 

^en<TTm-85 

u  m 


-  (4TT)*n[log(9D/9)X(CrTm>9})  -  9 “l  J  (^-9 )dX  +  0 [  (9-9D)2]  ] . 

{9  <TT  <9} 

Dm 


(20) 


Similarly  the  equality  in  the  middle  of  (18)  gives 


De(m) -D  -  (2tt)'"[(8  -eD)Mtnm>8])+  J  -  9  D>dX  1 .  (21) 

tVV9) 


We  thus  have 


(Rm(D0(m))  -  Rm(D))/(D0(m)  -  D)  +  2'n9‘1|  - 

(4rr)'n|  (log(9D/9)+9^L)X(CTTni>8])+0((8  -  9D)2)|  /  (DQ  (m)  -  D) 

<  2"n|  (log(9D/9)+8^1)/(6  -9D)  +  0(9  -  Qp)  A  ({^>9}  )|  .  (22) 


Since  lira  |  (log(9D/9)  +  9 ^ ^ / (9  -9D)|  *  0  and  X^tt^S^})  >  0,  the 
9 

limit  as  9  approaches  9^  of  the  right-hand  side  of  (22)  is  zero. 

Thus  R' (D)  -  -2-n9 " 1 ,  which  was  to  be  shown, 
m  D 

We  may  now  use  Lemma  4  to  prove  the  main  analytical  result  of  this 
paper.  In  particular,  we  have  the  following: 


Theorem  1:  Suppose  v  is  a  2-altemating  capacity  on  <p  ,3) .  Define  P  » 
(2rr)  nvf) )  and  P  »  (2tt)  n  f  rr  dX,  where  rr  ■  dv/dX.  Then  sup  R  (D)  is 

v  ft  v  v  *£7n  m 
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defined  parametrically  on  (0,Pv)  by  the  equations 


sup  R  (Da )  *  (4n)'n  f  max[0,log(rr  (u>)/9  )}X  (dcu) 


Dq  -  (2TT)-n^  mint9  >%(<“)}*■(<&“)» 


and  sup  R  (D)  =»  0  for  D  €  [P  ,P].  Moreover,  there  exists  a  measure 
n£/7!v  m  v 

q  6  >7?  such  that  sup  R  (D)  »  R  (D)  and  dq/dX  ■  rr  a.e.  [X]. 
v  rZ  m  q  ^  •  v  1 

Proof :  Suppose  m  6  ;  then  R^fl))  Is  absolutely  continuous  on  any  interval 

[D,P]  with  D>  0  since  it  is  convex  on  (0,P]  (Royden  [21,  p.  109]).  Thus, 

for  any  D  €  (0,P), 


P  P 

Rm(D)  "  Rm(p>  "  S  K (X)dX  *  'I  K^dX> 

m  m  tn  «_  m 

D  D 


where  R'  (x)  is  obtained  from  Lemma  4  forx€  (0,P  )  andR'  (x)  *0  for  x*  P  . 
m  m  m  m 


Equation  (25)  is  equivalent  to 


R  (D) 
m 


2-n  j  m  e-l(m)dx.  d€  (0,P  ) 
D 


;  D€  [P  P] 
m 


where  8x(m)  is  the  unique  value  of  9  solving  x  *  (2tt)  n[m((n^8})  + 
9X  ({^>9})  ] .  Note  that  the  existence  and  uniqueness  of  ®x(m)  follows 
from  the  development  in  the  proof  to  Lemma  4.  Suppose  q€$v  is  such 
that  dq/dX  «  tt^  a.e.  [X]  and  q (£tt^<  9 3  >  ■  v(CTTv.—  9]  )  for  all  8  2  0  (the 
existence  of  such  a  q  follows  from  Lemma  2).  Then 
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m  ( C  tt  ^9})+0X((n  >e})  »  inf  [m(BC) +9X  (B)  ]  <  inf  [v(BC)  +  9X  (B)] 
m  m  B6  6  B6  8 


V(CTTV—  9}  )  +9X  (CTTV>  © }  )  *  q({"q^6]  )  +0X({TTq>03). 


Thus,  since  [m({TT  ^93)+0X({tt  >0})]  is  strictly  increasing  in  9  on 
mm 

(  0,&  ),  we  must  have  9  (q)^9  (m)  for  all  x€  (0,P  ).  Therefore,  we 
m  x  x  m 

have  9  1(q)^9  *(m)  for  all  x€  (0,P  )  and,  since  Lemma  3  implies  P  i  P  , 
xx  m  q  m 

(26)  gives  R  (D)^  R  (D)  for  all  d€  (0,P).  To  complete  the  proof  we  need 
m  q 

only  note  that  the  quantity  defined  by  (23)  and  (24)  is  R  (D)  on  (0,P  ) 

q  q 

and  that  P  *  P  . 

q  v 

Theorem  1  implies  that  the  quantity  sup  R  (D)  is  the  rate-distortion 

nSU-sf  m 

curve  corresponding  to  the  spectral  density  TT^which  is  the  Huber-Strassen 

derivative  of  v  with  respect  to  Lebesgue  measure  on  Q  ,B) .  This  theorem 

also  states  that  there  is  a  q€fl?  that  has  distortion  rate  sup  R  (D)  for 

v  m 

in 

each  d€  (0,P].  Thus,  for  classes  of  the  form®?  ,  the  spectral  density 


tt^  represents  a  worst -case  or  least-favorable  spectrum  in  terms  of  MSE 

distortion  rate,  and  the  problem  of  finding  sup  R  (D)  is  solved  once  tt 

n07lv  m  V 

is  determined.  For  the  examples  given  in  Section  II,  the  solution  is 


thus  obtained.  In  general,  we  have  the  following  theorem  which  characterizes 
the  spectral  measures  q€®?v  (withdq/dX  ■  tt^  a.e.  [X] )  that  are  singled 
out  by  Lemma  2. 

Theorem  2 :  For  each  measure  m€®?v>  define  m'  to  be  the  part  of  m  that 
is  absolutely  continuous  with  respect  to  X  (via  the  Lebesgue  decomposition). 


A  measure  q€®?v  satisfies  the  conclusion  of  Lemma  2  if  and  only  if,  q 
minimizes  the  quantity 


(27) 


J (m)  -  j*  log[d(X  +  m')/dX]d(X  +  m') 

n 

over  ftly- 

Proof :  Suppose  q  satisfies  the  conclusion  of  Lemma  2,  i.e., 
dq/dX  *  rrv  a.e.  [X]  and  q({rrv  «  9})  *  v({ttv  *  Q})  for  all  0*0. 

Noting  that  7I\^  is  convex, define,  for  each  y6  [0,1]  and  m €^,  the  measure 
-  (1-Y)q+Ym  and  the  function  tt^  *  d(n^+X)/dX.  Since  xlog(x)  is 

convex  for  x€  (0,®)  and  since  nv'  ■  [(l-Y)q' +  Ym' ]  we  have  that  J(m)  is 

/ 

convex  on  Thus  a  sufficient  condition  for  q  to  minimize  J(m)  over 

is  that  be  nonnegative  for  every  satisfying 

J(m)<«.  Since  log[rr^ (id) ]rr^ (ou)  is  convex  in  Y  on  [0,1]  for  each 

u)  €  Q  ,  we  have 

(TT1  ‘TT0)(1  +  lOg(TT0))  2  Y~l0Tylog(TTy)  -  TT0log(TT0)) 

<  TT^Og^)  -  TT0log(TT0)  ,  a.e.  [X]  .  (28) 

The  left-most  quantity  of  (28)  is  bounded  below  a.e. [X]  and  the  right-most 
quantity  is  integrable  with  respect  to  X.  Thus  since  the  term  Y'^TTylogCn^) 
TT0log(TT0))  converges  monotonically  to  (tt^  -  tTq)  (1  +  log (ttq) )  as  Y  *  0, 
the  Monotone  Convergence  Theorem  (Royden  [21, p.  84])  implies  that 

e>J(mY)/&Y|Y-0  ^  [TTy  log  (tty )  ]  /&y|  ^dX 

-J  (^  -TT0)(l  +  log(TT0))dX 
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“  ^  (x  +  losC1 +TTv))dm'  -  £  (l  +  l°g(l+Trv))dq'.  (29) 

Since  m'([TTv<8})<  m({rTv<93)<  v({ttv<Q})  *  q({TTv<8})  =  q*  (£tTv<  9} ) 
for  all  9  ^  0,we  see  that  rr^  is  stochastically  smaller  under  q'  than 
under  m' .  Thus,  since  (1  +  log (1  +ttv) )  is  increasing  in  tt^,  the  quantity 
in  (29)  is  nonnegative,  which  is  sufficient  for  q  to  minimize  J(m). 

We  now  see  that  if  q  is  as  in  Lemma  2  it  minimizes  J(m)  over  77^. 

Suppose  also  minimizes  J(m)  over  77?^.  Then,  since  J(m)  is 

convex  on  771^,  J((l-Y)q+Yp)  must  be  constant  forY^  [0,1].  This  implies, 

2  2 

for  instance,  that  t  J((l-Y)q+Yp  )/&Y  |  q  *  0*  Applying  analysis 
similar  to  that  yielding  (29),  we  have 


&2J((1-Y)q+Yp)/&Y2L  0  -  J*  C  (tt  -TT  )2/n  ]dX 


where  rr^  »  dp/dX.  Equation  (30)  implies  that  tt^  *  a.e.  [X]  and, hence, 
that  p'  ■  q '  mod[X].  Thus  p  also  satisfies  the  conclusion  of  Lemma  2. 

This  completes  the  proof. 

Note  that  the  quantity  J(m)  of  (27)  is  a  measure  of  the  distance  or 
divergence  of  the  measure  (X-fm1)  from  X  (see,  for  example,  Kullback  [22]). 
Thus,  we  see  that  the  least-favorable  spectral  measure  q  is  that  which  is 
closest  to  Lebesgue  measure  in  this  sense.  Since  Lebesgue  measure  on  Q 
represents  white  noise  (which  corresponds  to  a  memoryless  source)  we  see 
that  q  is,  in  a  sense,  the  "most  memoryless"  element  of  77^.  Note  also  that 
Lemma  3  implies  that  q  has  the  maximum  possible  nonsingular  power  of  any 
measure  in  27?v.  These  phenomena  agree  well  with  the  intuitive  meaning  of 
the  rate-distortion  function  since,  in  general,  a  memoryless  source  requires 
the  highest  rate  for  a  given  degree  of  distortion  when  no  other  constraints 
are  placed  on  the  source  spectrum. 
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IV.  EXAMPLE  -  MIXTURE  CONTAMINATED  GAUSS-MARKOV  SOURCES 

In  this  section  we  use  a  specific  example  of  the  mixture  model  of 
Example  1  in  Section  II  to  illustrate  our  results.  In 

particular  we  consider  a  one-parameter  source  (n  =  1)  whose  spectral  measure 
m  has  a  nominal  first-order  Markov  spectral  density  with  degree  of  mixture 
uncertainty  e.  That  is,  we  have  m  ■  (l-e)mQ  +  eh  where  h  is  unknown, 
e  €  [0,1],  and  where  m^  is  given  by 


mQ(A)  »  P(l-r  )  J  (l-2rcos  (cu)  +  r  )  \  (dm)  ,  A  €  Q  , 

A 


with  | r|  S  1.  Recall  that  P  >  0  is  the  source  power.  This  nominal  source 

corresponds  to  a  Gaussian  source  with  autocorrelation  function 

E(XiXi_ric)  ■  P*rlkl  ;  k  €  Z.  Ue  will  assume  in  the  following,  that  P»1  and 


r  >  0. 


The  rate-distortion  function  corresponding  to  the  nominal  source  m^ 
is  discussed  in  [2,  pp.  113-115].  This  function  is  given  by 


Rm  "  7l°8  Kl“r2)/Dl 
m0  2 


for  0  <  D  s  (l-r)/(l+r),  and  parametrically  by 

%  <V  ’  1  +  V-  •  !  t*n‘1<i^c*n  <T» 


<V )  -  ^ «Tr>  -  £[Ci2  (2!,B  +  2y9  > 

-  C i2  <2^  )  -  CX2  (2ye  )  -  2ye  log  (r)  ] 
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for  (l-r)/(l+r)  s  D  2  1,  which  corresponds  to  (l-r)/(l+r)  £  8  £  (l+r)/(l-r). 


-1  [r2-l+0  G+r2' 


,  0  2  Xg  S  n, 


yQ  -  tan 


L  r  sin(Xg) 

1  -  r  cos(Xg)  ’  ° 


;  0  S  yQ  1!  tt/2, 


and  C^2  denotes  Clausen's  integral  defined  by 


Ci, (x)  ■  -  f  log  (2  sin  (t/2))dt. 
0 


(Note:  There  apparently  are  some  minor  errors  in  Eq.  (4.5.35)  of  [2]  which 
should  correspond  to  (34);  however,  (34)  follows  directly  from  (12)  and  Eq.  (39) 
on  p.  272  of  Lewin  [23].) 

Equation  (8)  specifies  the  worst-case  spectral  density  (i.e.,  the 
Huber-Strassen  derivative)  for  this  model;  in  particular, we  have 


rrv(iu)  ■  max{  c ' ,  (1-e)  (1-r2)  (l-2rcos  (tt>)  +  r2)  l} ,  uj€[-tt,tt]. 


where  c'  is  defined  below  (8).  Straightforward  analysis  yields  that  c'  is 
the  solution  to  the  equation 


(l-e)DL  (m_)|  +  e  ■  c'  . 

^  0  8  «c'/ (1-e) 


The  worst-case  spectral  density  of  (38)  is  illustrated  in  Fig.  1  for  the 
case  r».5  and  c  *  .25.  Note  from  this  illustration  that,  since 
~  J*  rrvdX»l,  it  follows  that  c'  is  monotonically  increasing  with  e  for 
(l-r)/(l+r)  *  c'  <  1, and  that  c'  will  be  identically  1  for  all  e  a  e 


where  (1-e  )(l+r)/(l-r)  *  1;  i.e.  s  *-  2r/(l+r).  Thus,  for  all 

max  max 

e  a  ®max»  worst-case  source  is  memory less  (i.e.,  white).  Note  also 
that,  for  general  s,  a  fraction 

J  X  (du>)  (40) 

"  {rr  =  c'} 

of  the  worst-case  source  power  can  be  thought  of  as  being  due  to  a  msmory- 
less  component.  This  fraction  along  with  the  value  of  c'  are  plotted  versus 
e  in  Fig.  2  for  the  case  r*.75  (for  which  6mav.  a  6  /  7 ) .  Note  that,  for 
e*.l,  about  12%  of  the  source  power  is  due  to  the  memory  less  component  and, 
for  e  *  .3, about  45%  is  due  to  the  memory less  component. 

The  rate-distortion  function  for  the  worst-case  source  can  be  derived 
straightforwardly  from  (12)  and  (13).  After  some  analysis  we  have 

R  (D)  -  jlog  (c'/D)  +  R  (DU(nu))|  (41) 

q  2  m0  9  0  9  *  c '  /  (1-e) 

for  0  <  D  i  c',  and  R^(D)  is  given  parametrically  by 

Ds  (q)  ■  (l-c)D  (m  )|  +e  (42) 

Y  U  V  “9/ (1-e) 

and 


■  V.  (VV>I 


-Vy  -  v -e/d-c) 


(43) 


for  c'  <  D  S  1,  which  corresponds  to  c'  <  9  i  (1-e) (1+r)/ (1-r) .  Here 
(m^)  and  R^  (D^On^))  are  defined  by  (33)  through  (37)  and  q  denotes  the 
measure  corresponding  to  the  Lebesgue  density  ttv  (i.e.,  dq  =  TT^dX ) .  The 
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rate-distortion  function  of  (41)  through  (43)  is  plotted  in  Fig.  3  for  the 

case  r  ■  .75  and  for  several  values  of  e.  Note  that,  as  s  increases  from 

zero  to  e  ,  the  convexity  of  the  worst-case  curve  becomes  less  pronounced.  Of 

course,  the  e  =  0  curve  corresponds  to  the  uncontaminated  Gauss-Markov  source,  and 

the  e  2  c  curve  corresponds  to  a  memory  less  source  which  is  universally  the  worst  case. 

It  is  interesting  to  note  that  (39)  and  (41)  through  (43)  also  apply 
to  mixtures  with  nominal  spectra  other  than  the  Gauss-Markov.  That  is,  if 
^TTgdX  =  2tt  where  tTq  =  dm^/dX,  then  the  mixture  model  with  nominal  measure 
mg  has  worst-case  spectral  density  given  by  (8)  and  (39),  and  the  corres¬ 
ponding  rate-distortion  function  is  given  by  (41)  through  (43)  with 


D^(mg)  and  (D^(nig))  being  the  rate-distortion  equations  for  mg.  Thus, 
behavior  similar  to  that  of  Fig.  3  would  be  expected  for  other  mixtures. 


For  example,  for  e  £  1  -  ess  sup  TTn(ui),  such  a  mixture  class  would  contain 

u)€  Q 

a  memory less  source. 


V.  EXTENSION  TO  BAND  LIMITED  CONTINUOUS -PARAMETER  SOURCES 


Theorems  1  and  2,  as  proven  above,  apply  to  discrete-parameter  sources. 

Rate-distortion  functions  for  continuous -parameter  sources,  however,  can 

also  be  determined  from  (12)  and  (13)  provided  their  spectral  densities 

are  essentially  bounded  (see  Berger  [2,  pp.  116-122]).  Thus,  if  we  assume 

that  X  *  {X£;  t  €  Rn}  is  a  homogeneous  continuous-parameter  source  with 

spectral  measure  m  €  171^,  a  capacity  class  on  (£n,/?),  we  should  be  able  to 

prove  results  analogous  to  Theorems  1  and  2.  This  is  the  case  if  we  make 

c  n 

the  additional  restriction  that  v(fi  )  =  0  for  some  compact  Q  c  S  ,  This 
restriction  is  equivalent  to  assuming  that  all  sources  have  a  contnon 
(finite)  bandlimit  and  is  sufficient  to  assure  that  (12)  and  (13)  apply  to 
all  source  spectra  in  771  and  to  apply  the  existing  Huber-Strassen  theory 
to  differentiate  v  with  respect  to  Lebesgue  measure  on  Q.  We  thus  have 
Theorem  3:  Theorems  1  and  2  hold  for  n-continuous -parameter  sources 
provided  Q  is  a  compact  subset  of  Rn. 

It  should  be  noted  that  Huber-Strassen  derivatives  of  capacities  with 
respect  to  o-finite  (and  not  finite)  measures  can  be  constructed  [24] ;  thus 
Theorem  3  can  possibly  be  extended  to  nonbandlimited  sources  provided  that 
the  definition  of  tt^  is  appropriately  modified.  However,  several  of  the 
most  useful  examples  of  capacity  classes  (e.g.,  the  e-mixtures  and  varia¬ 
tion  neighborhoods)  fail  to  be  capacity  classes  when  Q  is  not  compact. 
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List  of  Footnotes 


1.  For  compactness  of  notation  we  will  write  £f  >  9}  to  denote  fio€  ft  |  f(tu)>9}. 

2.  Note  that  this  set  function  (and  the  one  defined  in  Example  2)  is  dis¬ 
continuous  from  above  at  the  null  set  0.  However,  for  Q  *  [-rr,TT]n,  this 
discontinuity  does  not  violate  the  property  that  v  must  be  continuous 
from  above  on  closed  sets  (i.e..  Property  (iv))  since,  in  a  compact 
separable  metric  space,  there  is  no  sequence  of  closed  sets  converging 
down  to  the  null  set  (Dunford  and  Schwartz  [11,  pp.  30-31]). 


List  of  Figures 


Fig.  1  -  Worst-case  spectral  density  for  a  contaminated -mixture  class 
(e  3  .25)  with  a  nominal  Gauss-Markov  source  (r*.5). 

Fig.  2  -  Worst-case  spectrum  parameter  c'  and  fraction  of  worst-case 
spectrum  power  contained  in  memory less  component  versus  e;  mixture- 
contaminated  Gauss-Markov  source  (r*.75). 


Fig.  3  -  sup  R  (D)  for  mixture-contaminated  Gauss-Markov  source  (r  » . 
m 


Frequency,  u> 


Fig.  1  -  Worst-case  spectral  density  for  a  contaminated- 
-  mixture  class  (s  *  .25)  with  a  nominal  Gauss - 
Markov  source  (r  *  .5). 


Nominal  Source: 
Gauss-Markov,  r=0.75 


Mixture  Contamination 


Worst- Case  Information  Rate,  R(nats) 


Fig 


.  3  -  sup  R  (D)  for  mixture -contaminated  Gauss-Markov 
o€  ° 

source  (r  ■  .75). 


